Fall detection system using a combination of accelerometer, audio input and magnetometer

ABSTRACT

A wearable device for detecting a user state is disclosed. The wearable device includes an accelerometer for measuring an acceleration of a user, a magnetometer for measuring a magnetic field associated with the user&#39;s change of orientation, a microphone for receiving audio, a memory for storing the audio, and at least one processor communicatively connected to the accelerometer, the magnetometer, the microphone, and the memory. The processor is identified to declare a measured acceleration as a suspected user state, and to categorize the suspected user state based on the stored audio as one of an activity of daily life (ADL), a confirmed user state, or an inconclusive event.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 61/516,479, filed Apr. 4, 2011, entitled “DISTRIBUTEDSYSTEM TO CLASSIFY HUMAN ACTIVITY ACROSS A WAN,” U.S. Provisional PatentApplication No. 61/516,480, filed Apr. 4, 2011, entitled “CLOUD BASEDMOBILE EMERGENCY CALL INITIATION AND HANDLING,” and U.S. ProvisionalPatent Application No. 61/404,379, filed Oct. 4, 2010, entitled “FALLDETECTION SYSTEM USING A COMBINATION OF ACCELEROMETER, AUDIO INPUT ANDMAGNETOMETER,” disclosures of which are incorporated herein by referencein its entirety.

TECHNICAL FIELD

Embodiments of the present invention relate generally to healthcare-based monitoring systems, and more particularly, to a system andmethod for detecting a predefined state of a user.

BACKGROUND

For certain age groups, such as the elderly, or people that engage incertain dangerous activities, such as firefighters and soldiers, a fallcan adversely affect health. As a result, many fall detection systemsand devices have been developed. Many such systems and devices employaccelerometers that measure sudden changes in acceleration that mayindicate a fall, such as rapid changes in acceleration followed by nomovement (i.e., lying on the floor). Such methods have difficultydistinguishing falls from activities of daily living (ADL). This makesit difficult to distinguish real falls from certain fall-like activitiessuch as sitting or lying down quickly, resulting in many falsepositives. Body orientation is also used as a means of detecting falls,but it is not very useful when the ending position is not horizontal,e.g., falls happening on stairs.

U.S. Patent Application Publication No. US 2006/0279426 A1 (hereinafter“the '426 publication”) describes a device which includes a user-wornaccelerometer and magnetometer that assumes a person is in a standingposition. A fall event is declared when a significant and rapidacceleration signal coincides with a shift in ambient magnetic fieldsbetween two levels. However, the device of the '426 publication requirescomplicated algorithms to remove false positives and negatives, and istherefore computationally expensive, power hungry, and producesuncertain results.

A paper by Q. Li, et al., titled, “Accurate, Fast Fall Detection UsingGyroscopes and Accelerometer-Derived Posture Information,” College ofWilliam and Mary, (hereinafter “Li et al.”) describes a system andmethod for detecting falls that employs gyroscopes in addition toaccelerometers. In Li et al., human activities are divided into twocategories: static postures and dynamic transitions. By using twotri-axial accelerometers at separate body locations, the system canrecognize four kinds of static postures: standing, bending, sitting, andlying. Motions between these static postures are considered to bedynamic transitions. Linear acceleration and angular velocity aremeasured to determine whether motion transitions are intentional. If thetransition before a lying posture is not intentional, a fall event isdeclared.

The system of Li et al. requires sensors to be distributed in severallocations, which is not convenient for a user nor is it practical toimplement when the user is in an unfamiliar environment. Moreover,continuous monitoring with gyroscopes requires a large amount of power.

Another conventional solution includes a stationary device that basesfall decisions on measurements of floor vibrations and audio analysis.To enable a user to have an audio conversation using a mobile version ofthe aforementioned stationary device, two microphones are employed toremove background noise. This solution arbitrarily designates onemicrophone to be the primary microphone for measurements and the othermicrophone is employed for detecting the background noise. This rendersit difficult to distinguish between human activity and other sources ofnoise vibration, such as an object falling off a table.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be more readily understoodfrom the detailed description of exemplary embodiments presented belowconsidered in conjunction with the attached drawings in which likereference numerals refer to similar elements and in which:

FIG. 1 depicts an exemplary system for detecting a fall, according to anembodiment of the present invention;

FIG. 2 is a block diagram of the components of the wearable deviceemployed in the system of FIG. 1, according to an embodiment of thepresent invention;

FIG. 3A is a flow diagram of one embodiment of a method for detecting afall using the wearable device of FIG. 2;

FIG. 3B is a flow diagram of another embodiment of a method fordetecting a fall using the wearable device of FIG. 2; and

FIG. 4 is a block diagram of a representative classification method thatmay be employed to train and operate one or more classifiers forre-confirming a suspected fall, according to an embodiment of thepresent invention.

DETAILED DESCRIPTION

Embodiments of the invention provide a wearable device configured todetect a predefined state of a user. The predefined state may include auser physical state (e.g., a user fall inside or outside a building, auser fall from a bicycle, a car incident involving a user, etc.) or anemotional state (e.g., a user screaming, a user crying, etc.). Thewearable device may include an accelerometer for measuring anacceleration of the user, a magnetometer for measuring a magnetic fieldassociated with the user's change of orientation, a microphone forreceiving audio, a memory for storing the audio, and a processing device(“processor”) communicatively connected to the accelerometer, themagnetometer, the microphone, and the memory. The processor periodicallyreceives measurements of acceleration and/or magnetic field of the userand stores the audio captured by the microphone in the memory. Theprocessor is configured to declare a measured acceleration and/or acalculated user orientation change based on the measured magnetic fieldas a suspected user state. The processor may then categorize thesuspected user state based on the stored audio as an activity of dailylife (ADL), a confirmed predefined user state, or an inconclusive event.

In one embodiment, the wearable device further comprises a gyroscopecommunicatively connected to the processor, where the processor isconfigured to calculate a change of orientation of the user from thegyroscope, the magnetometer, and accelerometer that is more accuratethan a change of orientation calculated from the magnetometer andaccelerometer alone. The wearable device may further comprise a speakerand a cellular transceiver each communicatively connected to theprocessor, where the processor is configured to employ the speaker, themicrophone, and the cellular transceiver to receive a notification andan optional confirmation from a voice conversation with a call center orthe user.

In one embodiment, the processor is further configured to extract atleast one feature from the stored audio and the measured accelerationand/or magnetic field. The feature may be a time domain, frequencydomain or an inter-signal dynamic property. The inter-signal dynamicproperty may be based on relationships between audio energy and physicalmovement. The inter-signal dynamic property may be elapsed time betweenacceleration and audio peaks or between acceleration and rotation ratepeaks.

In one embodiment, the wearable device further comprises a cellulartransceiver configured to communicate with a cloud computing system,where the processor is operable to employ the cellular transceiver totransmit the stored audio and the measured acceleration and/or magneticfield and/or the calculated change of orientation to the cloud computingsystem and receive a re-confirmation or change of classification fromthe cloud computing system based on the stored audio and the measuredacceleration and/or magnetic field and/or the calculated change oforientation. The re-confirmation or change of classification may bebased on output of a trained classifier operable to render a decisionbased on the stored audio and/or the measured acceleration and/ormagnetic field and/or the calculated change of orientation. The trainedclassifier may be a combination of a Gaussian Mixture model (GMM) forclassifying falls and a GMM for classifying ADLs.

FIG. 1 depicts an exemplary system 10 for detecting a predefined userstate, according to an embodiment of the present invention. The system10 includes wearable devices 12 a-12 n communicatively connected to adistributed cloud computing system 14. A wearable device 12 may be asmall-size computing device that can be wearable as a watch, a pendant,a ring, a pager, or the like, and can be held in multiple orientations.

In one embodiment, each of the wearable devices 12 a-12 n is operable tocommunicate with a corresponding one of users 16 a-16 n (e.g., via amicrophone, speaker, and voice recognition software), external healthsensors 18 a-18 n (e.g., an EKG, blood pressure device, weight scale,glucometer) via, for example, a short-range OTA transmission method(e.g., BlueTooth), and the distributed cloud computing system 14 via,for example, a long range OTA transmission method (e.g., over a 3G or 4Gcellular transmission network 20). Each wearable device 12 is configuredto detect predefined states of a user. The predefined states may includea user physical state (e.g., a user fall inside or outside a building, auser fall from a bicycle, a car incident involving a user, a user takinga shower, etc.) or an emotional state (e.g., a user screaming, a usercrying, etc.). As will be discussed in more detail below, the wearabledevice 12 may include multiple sensors for detecting predefined userstates. For example, the wearable user device 12 may include anaccelerometer for measuring an acceleration of the user, a magnetometerfor measuring a magnetic field associated with the user's change oforientation, and a microphone for receiving audio. Based on datareceived from the above sensors, the wearable device 12 may identify asuspected user state, and then categorize the suspected user state as anactivity of daily life, a confirmed predefined user state, or aninconclusive event. The wearable user device 12 may then communicatewith the distributed cloud computing system 14 to obtain are-confirmation or change of classification from the distributed cloudcomputing system 14.

Cloud computing provides computation, software, data access, and storageservices that do not require end-user knowledge of the physical locationand configuration of the system that delivers the services. The term“cloud” refers to a plurality of computational services (e.g., servers)connected by a computer network.

The distributed cloud computing system 14 may include one or morecomputers configured as a telephony server 22 communicatively connectedto the wearable devices 12 a-12 n, the Internet 24, and one or morecellular communication networks 20, including, for example, the publiccircuit-switched telephone network (PSTN) 26. The distributed cloudcomputing system 14 may further include one or more computers configuredas a Web server 28 communicatively connected to the Internet 24 forpermitting each of the users 16 a-16 n to communicate with a call center30, first-to-answer systems 32, and care givers and/or family 34. Thedistributed cloud computing system 14 may further include one or morecomputers configured as a real-time data monitoring and computationserver 36 communicatively connected to the wearable devices 12 a-12 nfor receiving measurement data, for processing measurement data to drawconclusions concerning a potential predefined user state, fortransmitting user state confirmation results and other commands back tothe to the wearable devices 12 a-12 n, and for storing and retrievingpresent and past historical predefined user state feature data from adatabase 37 which may be employed in the user state confirmationprocess, and in retraining further optimized and individualizedclassifiers that can in turn be transmitted to the wearable device 12a-12 n.

FIG. 2 is a block diagram of the components of an exemplary wearabledevice 12 a employed in the system of FIG. 1, according to an embodimentof the present invention. The wearable device 12 a may include alow-power processor 38 communicatively connected to an accelerometer 40(e.g., a 3-axis accelerometer) for detecting acceleration events (e.g.,high, low, positive, negative, oscillating, etc.), a magnetometer 42(preferably a 3-axis magnetometer), for assessing a magnetic field ofthe wearable device 12 a, and an optional gyroscope 44 for providing amore precise short term determination of orientation of the wearabledevice 12 a. The low-power processor 38 is configured to receivecontinuous or near-continuous real-time measurement data from theaccelerometer 40, the magnetometer 42, and the optional gyroscope 44 forrendering tentative decisions concerning predefined user states. Byutilizing the above components, the wearable device 12 is able to renderthese decisions in relatively low-computationally expensive, low-poweredmanner and minimize false positive and false negative errors. A cellularmodule 46, such as the 3G IEM 6270 manufactured by QCOM, includes ahigh-computationally-powered microprocessor element and internal memorythat are adapted to receive the suspected fall events from the low-powerprocessor 38 and to further correlate orientation data received from theoptional gyroscope 44 with digitized audio data received from one ormore microphones 48 (preferably, but not limited to, amicro-electro-mechanical systems-based (MEMS) microphone(s)). The audiodata may include the type, number, and frequency of sounds originatingfrom the user's voice, the user's body, and the environment.

The cellular module 46 is also configured to receive commands from andtransmit data to the distributed cloud computing system 14 via a 3G or4G transceiver 50 over the cellular transmission network 20. Thecellular module 46 is further configured to communicate with and receiveposition data from an aGPS receiver 52, and to receive measurements fromthe external health sensors 18 a-18 n via a short-range BlueToothtransceiver 54. In addition to recording audio data for event analysis,the cellular module 46 is further configured to permit direct voicecommunication between the user 16 a and the call center 30,first-to-answer systems 32, or care givers and/or family 34 via abuilt-in speaker 58 and an amplifier 60. The cellular module 46 mayreceive/operate a plurality of input and output indicators 62 (e.g., aplurality of mechanical and touch switches (not shown), a vibrator,LEDs, etc.). The wearable device 12 a also includes an on-board batterypower module 64. The wearable device 12 a may also include emptyexpansion slots (not shown) to collect readings from other internalsensors (i.e., an inertial measurement unit), for example, a pressuresensor (for measuring air pressure, i.e., attitude) or heart rate, bloodperfusion sensor, etc.

FIG. 3A is a flow diagram of one embodiment of a method for detecting afall. The method is performed by processing logic that may comprisehardware (circuitry, dedicated logic, etc.), software (such as is run ona general purpose computer system or a dedicated machine), or acombination of both. In one embodiment, the method is performed by auser device (e.g., wearable device 102 of FIG. 1).

Referring to FIGS. 1, 2 and 3A, at block S1, the low-power processor 38periodically scans the accelerometer 40. If the low-power processor 38detects at block S2 a large negative acceleration of a user (i.e., anacceleration below a certain threshold, which can be individuallyoptimized for the user, and controlled by the distributed cloudcomputing system 14 or the processor 38), then a “suspected fall” eventis declared by the low-power processor 38; otherwise, the method returnsto block S1. If a “suspected fall” is declared, control may betransferred to the more computationally-intensive high-power processorelement within the cellular module 46. At block S3, the cellular module46 records and stores digitized audio received from the microphone(s) 48in its internal memory (not shown) for a predetermined amount of time.The audio data recorded may begin either from the moment the suspectedfall is initially detected or from a few seconds before the beginning ofthe suspected fall if the audio data was stored in a continuous bufferin an internal memory of the cellular module 46. Optionally at block S4,the cellular module 48 may activate the gyroscope 44 to obtain (eitherdirectly or via the low-power processor 38) samples of more accurateorientation change data.

In an embodiment, the gyroscope 44 is not optional, but automaticallyactivated by the cellular module 48 to obtain samples of more accurateorientation change data. The gyroscope 44 may be used to filter datareceived from the magnetometer 42 and accelerometer 40 to achieve a moreaccurate orientation calculation to confirm or reject “suspected fall”events based on a predetermined minimum change in orientation.

At block S5, the cellular module 48 employs at least one audioprocessing algorithm to confirm the “suspected fall” event. Exemplaryaudio processing algorithms may include, but are not limited to, anopt-out mechanism in which specific sound patterns are used to assessthe “suspected fall” as normal (e.g., complete silence, talking,walking, etc., collectively “activities of daily life” or ADL events),or an opt-in mechanism in which the cellular module 48 detects specificsounds such as a “bang” or a “scream” to confirm that the fall hashappened. When an opt-in mechanism is employed, specific features of theaudio data may be extracted, which may be based on relationships betweenaudio energy and physical movement of the user 16 a (via the wearabledevice 12 a). An exemplary relationship may include, but is not limitedto, elapsed time between acceleration and audio energy peaks. Exemplaryfeatures may include, but are not limited to, time domain propertiessuch as vertical velocity, rotation angle, and vertical distance peaks;frequency domain properties such as spectral envelope, dominantfrequency, and periodicity measurement; signal dynamics properties inboth time and frequency domain, such as changes in signal amplitude andfrequency content over time; and inter-signals dynamic properties suchas elapsed time between acceleration and audio peaks or acceleration androtation rate peaks.

These features may be directly extracted from measurement data by thecellular module 48 or may be passed on to the real-time data monitoringand computation server 36 of the distributed cloud computing system 14via the 3G/4G transceiver 50 and the cellular transmission network 20.The real-time data monitoring and computation server 36 may then extractthe features from the measurement data and may then return the extractedfeatures to the cellular module 48 for classification or directlyperform the classification and return classification results (i.e., aconfirmed fall event or an ADL) to the cellular module 48.

Additionally, other sensors may be employed in the decision process,which may include data from the health sensors 18 a-18 n received overthe short-range BlueTooth transceiver 54 and the aGPS receiver 52. Theinformation of some or all of the sensors may be used together tovalidate the suspected fall. Different weighting mechanisms may beapplied dynamically (e.g., using a neural net algorithm). An exemplarytable of weights versus classification is shown in Table 1 as follows:

TABLE 1 Accelerometer Gyroscope Voice Result 60% No 70% No 60% Yes 50%Yes 50% Yes 40% Re-analyze Accelerometer and voice data 70% No 20% No

If, at block S6, a suspected fall is classified as a “confirmed fall”,then at block S7, the confirmed fall is reported to the call center 30,the first-to-answer systems 32, or care givers and/or family 34, who mayfurther assess the “confirmed” fall by listening in or replaying audiodata returned on command to/from the wearable device 12 a via the Webserver 28. Conversations with the user 16 a may be employed to identifya false positive or a false negative.

The decision table (i.e., Table 1) may be dynamically adaptive in thesense that the system 10 may “learn” from past incidents of confirmedactivities. The measurement data from past confirmed activities may bestored on the wearable device 12 a or in the database 37 associated withthe data monitoring and computation server 36 and later used if falseclassification was confirmed through the call center 30.

If, at block S6, the classification of the suspected fall isinconclusive, then at block S8, stronger algorithms may be applied basedon sensor data transmitted to the real-time data monitoring andcomputation server 36 of the distributed cloud computing system 14 (as aresult, power may be conserved in the wearable device 12 a byrestricting the low-power processor 38 and/or the cellular module 48 torelatively simple initial assessment algorithms). A person skilled inthe art would appreciate that the computational power of embeddedprocessors may continue to rapidly improve over time and likewise thecost of such devices may decrease over time such that the completemethod described in FIG. 3A may be executable on a single embeddedprocessor incorporated within the wearable device 12 a.

Returning to block S8, the decision whether to send raw measurement dataor extracted features to the data monitoring and computation server 36may be made dynamically within the wearable device 12 a or may be madeby the data monitoring and computation server 36 (i.e., raw data may bestored on the wearable device 12 a for period of time). If the datamonitoring and computation server 36 sends a command to the wearabledevice 12 a to transmit raw measurements, the data monitoring andcomputation server 36 may extract desired features directly from the rawmeasurements. The data monitoring and computation server 36 may employon-demand cloud computing to exploit massive parallelism and GPUcomputing to return a more accurate classification in real-time to thewearable device 12 a.

Once the data monitoring and computation server 36 receives or computesfeatures, a re-confirmation of a suspected fall or a re-classificationof the activity as an ADL may takes place (i.e., determining whether a“suspected fall” is a confirmed fall, stumble, or normal ADL, and if so,what kind of fall or ADL, and how severe). There-confirmation/re-classification may be fully automatic, or in somecases, decided by a human operator based on the computed features orbased on a conversation with the user 16 a via execution of avoice-to-text algorithm, the speaker 58, and the microphone(s) 48 on thewearable device 12 a, and/or via an interactive voice response system(IVR) integrated with the Web server 28.

The stronger algorithms that may be employed include, but are notlimited to, the training and testing of classifiers. A person skilled inthe art would recognize that any suitable trained classifier may beemployed, including, but not limited to, a Gaussian mixture model (GMM)or a support vector machines (SVM), as is known in the art. In oneembodiment, two GMMs are employed: one trained on positive samplesindicative of a fall and one trained on negative samples indicative ofan ADL.

FIG. 3B is a flow diagram of another embodiment of a method fordetecting a fall. Referring to FIGS. 1, 2 and 3B, at block T1, thelow-power processor 38 periodically scans the accelerometer 40 (e.g., atri-axial accelerometer) to receive acceleration signals, a_(y), a_(y),a_(z), along three axis (e.g., x, y, and z in a rectangular coordinatesystem) of the accelerometer 40.

At block T2, the individual acceleration signals, a_(y), a_(y), a_(z),are pre-processed by means of filtering and smoothing methods known inthe art. At block T3, the preprocessed acceleration signals are combinedinto a total acceleration signal, a_(tot), according to Equation 1:

a _(tot)=√{square root over ((a _(x))²+(a _(y))²+(a _(z))²)}{square rootover ((a _(x))²+(a _(y))²+(a _(z))²)}{square root over ((a _(x))²+(a_(y))²+(a _(z))²)}  (1)

If the low-power processor 38 detects at block T4 a low acceleration ofa user (i.e., a value of final acceleration a_(tot), below an adaptablethreshold Th_(L)), then a “suspected fall” event is declared by thelow-power processor 38; otherwise, the method returns to block T1.

If a “suspected fall” is declared, control may be transferred to themore computationally-intensive high-power processor element within thecellular module 46 where, at blocks T5 a, T5 b, T5 c, the cellularmodule 48 may activate the magnetometer 42, the microphone(s) 48 (i.e.,cellular module 48 records and stores digitized audio received from themicrophone(s) 48 in its internal memory (not shown)), and the gyroscope44 (i.e., to obtain (either directly or via the low-power processor 38)samples of more accurate orientation change data), for a predeterminedamount of time.

At block T6, the raw measurement data received from the accelerometer40, the magnetometer 42, the microphone(s) 48, and the gyroscope 44 bythe cellular module 48 are transmitted to the real-time data monitoringand computation server 36 of the distributed cloud computing system 14via the 3G/4G transceiver 50 and the cellular transmission network 20.At block T7, the data monitoring and computation server 36 may extractdesired features directly from the raw measurements. The data monitoringand computation server 36 may employ on-demand cloud computing toexploit massive parallelism and GPU computing to return a more accurateclassification in real-time to the wearable device 12 a.

Once the data monitoring and computation server 36 computes features, are-confirmation of a suspected fall, a re-classification of the activityas an ADL, or an inconclusive event may takes place (i.e., determiningwhether a “suspected fall” is a confirmed fall, stumble, or normal ADL,and if so, what kind of fall or ADL, and how severe). There-confirmation/re-classification may be fully automatic, or in somecases, decided by a human operator based on the computed features orbased on a conversation with the user 16 a via execution of avoice-to-text algorithm, the speaker 58, and the microphone(s) 48 on thewearable device 12 a, and/or via an interactive voice response system(IVR) integrated with the Web server 28.

As described hereinabove, trained and tested classifiers may be employedin the re-confirmation process, which may include, but are not limitedto, a Gaussian mixture model (GMM) or a support vector machines (SVM),as is known in the art. In one embodiment, two GMMs are employed: onetrained on positive samples indicative of a fall and one trained onnegative samples indicative of an ADL.

If, at block T7, a suspected fall is classified as a confirmedfall/inconclusive event, then at block T8, the confirmedfall/inconclusive event is reported to the call center 30, thefirst-to-answer systems 32, or care givers and/or family 34, who mayfurther assess the “confirmed” fall by listening in or replaying audiodata returned on command to/from the wearable device 12 a via the Webserver 28. Conversations with the user 16 a may be employed to identifya false positive or a false negative. If, at block T8, a suspected fallis classified as an ADL, then processing returns to block T1.

FIG. 4 is a block diagram of a representative classification method thatmay be employed to train and operate one or more classifiers, accordingto an embodiment of the present invention. The method comprises twophases: a training stage 60 and a testing stage 62. The purpose of thetraining stage 60 is to train a model for each group of events: falls 70and ADLs 72. The input to the training stage 60 is a set of fall and ADLfeatures previously derived from measurement sample data (e.g., fromhuman volunteers) of events from each group. The input measurementsamples undergo pre-processing and feature extraction. Then, a featuresubset that best differentiates between the groups is selected, and aGMM model for each group is trained. The outputs of the training stage60 are a set of indices, which represents the selected feature subsetand the two GMM models 70, 72. These outputs are stored in the database37 associated with the data monitoring and computation server 36.

More particularly, measurement data from prior confirmed fall and ADLevents received from either in-use or experimental subjects are receivedfrom the plurality of wearable devices 12 a-12 n (i.e., a form ofadapting and system learning built into the system 10) into a portion ofthe database 37 designated as a fall and ADL events database 64. Thedata is processed and relevant features are extracted from each event ina feature extraction block 66. Gaussian mixture model (GMM) classifiersare implemented on a subset of a selected features space 68, selectedvia a sequential forward floating selection (SFFS) algorithm designed tomaximize classifier performance. Each class, fall 70 and ADL 72, isrepresented by a GMM trained using an expectation maximization algorithm71.

In the testing stage 62, an unknown event is introduced to each of theclassifiers 70, 72 (i.e., when a new event is recorded by the wearabledevice 12 a that is considered to be an inconclusive event). Theinconclusive event undergoes pre-processing, and the previously selectedfeatures are extracted in block 74 and saved as a feature matrix. Ascore is calculated for the inconclusive event in a model matching phase76. The classification decision 78 is based on a comparison of the scorewith a threshold 80 determined during the training phase 60 as either aconfirmed fall event 82, an ADL event 84, or an inconclusive event (notshown).

It should be noted that numerous variations of mechanisms discussedabove can be used with embodiments of the present invention without lossof generality. For example, a person skilled in the art would alsoappreciate that the complete method described in FIGS. 3A and 3B may beexecuted on a single embedded processor incorporated within the wearabledevice 12 a. A person skilled in the art would also appreciate that, inaddition to inconclusive events, measurement data and recorded audioand/or extracted features of initially confirmed falls and/or ADLs maybe passed on to the trained classifiers 70, 72 for a final decision(i.e., a re-confirmation of a fall or a re-classification as an ADL).

The combination of sensors may be employed to detect predefined userstates in various environments and during various activities such as butnot limited to: inside a house, while sleeping, while walking orexercising outside the house, while in the shower, etc. The combinationof sensors may be used to detect a “shower mode,” wherein microphone andspeaker levels are automatically adjusted to filter out ambient waternoise. This may be achieved by running a simple algorithm on themicrophone 48 output to detect that the user 16 a is in the shower. Whenthe user 16 a is in a “shower mode,” the wearable device 12 a may employa different algorithm and decision table to detect possible falls. Asanother example, the wearable device 12 a can differentiate between afall in the shower and a fall outside the shower. The sensor measurementthresholds employed in the decision table (i.e., Table 1) would providedifferent results when the user 16 a is in “shower mode.”

Returning to FIG. 2, the device 12 a may also include a main memory(e.g., read-only memory (ROM), flash memory, dynamic random accessmemory (DRAM) such as synchronous DRAM (SDRAM)), a static memory (e.g.,flash memory, static random access memory (SRAM)), and a data storagedevice, which communicate with each other and the processor 38 via abus. Processor 38 may represent one or more general-purpose processingdevices such as a microprocessor, distributed processing unit, or thelike. More particularly, the processor 38 may be a complex instructionset computing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,or a processor implementing other instruction sets or processorsimplementing a combination of instruction sets. The processor 38 mayalso be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. The processor 38 is configured to perform the operationsand/or functions discussed herein.

The user device 38 may further include a video display unit (e.g., aliquid crystal display (LCD) or a cathode ray tube (CRT)), an inputdevice (e.g., a keyboard or a touch screen), and a drive unit that mayinclude a computer-readable medium on which is stored one or more setsof instructions embodying any one or more of the methodologies orfunctions described herein. These instructions may also reside,completely or at least partially, within the main memory and/or withinthe processor 38 during execution thereof by the user device 12 a, themain memory and the processor also constituting computer-readable media.

The term “computer-readable storage medium” should be taken to include asingle medium or multiple media (e.g., a centralized or distributeddatabase, and/or associated caches and servers) that store the one ormore sets of instructions. The term “computer-readable storage medium”shall also be taken to include any medium that is capable of storing,encoding or carrying a set of instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies discussed herein. The term “computer-readable storagemedium” shall accordingly be taken to include, but not be limited to,solid-state memories, optical media, and magnetic media.

In the above description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that embodiments of the invention may bepracticed without these specific details. In some instances, well-knownstructures and devices are shown in block diagram form, rather than indetail, in order to avoid obscuring the description.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “identifying”, “categorizing”, “receiving”, “extracting”or the like, refer to the actions and processes of a computer system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (e.g., electronic) quantities within thecomputer system's registers and memories into other data similarlyrepresented as physical quantities within the computer system memoriesor registers or other such information storage, transmission or displaydevices.

Embodiments of the invention also relate to an apparatus for performingthe operations herein. This apparatus may be specially constructed forthe required purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear from the description below.In addition, the present invention is not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the invention as described herein.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the invention should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

1. An apparatus wearable by a user for detecting a user state,comprising: an accelerometer for measuring an acceleration of the user;a magnetometer for measuring a magnetic field associated with the userand calculation of the user's orientation; a microphone for receivingaudio; a memory for storing the audio; and at least one processorcommunicatively connected to the accelerometer, the magnetometer, themicrophone, and the memory, and configured to: identify a measuredacceleration and magnetic field as a suspected user state; andcategorize the suspected user state based on the stored audio as one ofan activity of daily life (ADL), a confirmed user state, and aninconclusive event.
 2. The apparatus of claim 1, wherein categorizingthe suspected user state based on the stored audio comprises an opt-outmechanism wherein specific sound patterns are used to assess thesuspected user state as an ADL.
 3. The apparatus of claim 1, whereincategorizing the suspected user state based on the stored audiocomprises an opt-in mechanism wherein specific sound are detected andemployed to confirm that a fall has happened.
 4. The apparatus of claim1, further comprising a gyroscope communicatively connected to the atleast one processor, wherein the processor is configured to calculate achange of orientation of the user from the gyroscope, the magnetometer,and accelerometer that is more accurate than a change of orientationcalculated from the magnetometer and accelerometer alone.
 5. Theapparatus of claim 1, further comprising: a speaker and a cellulartransceiver each communicatively connected to the at least oneprocessor, wherein the at least one processor is configured to employthe speaker, the microphone, and the cellular transceiver to receive anotification and an optional further confirmation from a voiceconversation with a call center or the user.
 6. The apparatus of claim4, wherein the at least one processor is further configured to extractat least one feature from the stored audio and at least one of themeasured acceleration or the measured magnetic field.
 7. The apparatusof claim 6, wherein the at least one feature is an inter-signal dynamicproperty.
 8. The apparatus of claim 7, wherein the inter-signal dynamicproperty is based on relationships between audio energy and physicalmovement.
 9. The apparatus of claim 8, wherein the inter-signal dynamicproperty is one of elapsed time between acceleration and audio peaks andacceleration and rotation rate peaks.
 10. The apparatus of claim 4,further comprising a cellular transceiver configured to communicate witha cloud computing system, wherein the at least one processor is operableto employ the cellular transceiver to: transmit the stored audio and atleast one of the measured acceleration and calculated change oforientation to the cloud computing system; and receive a re-confirmationor change of classification from the cloud computing system based on thestored audio and at least one of the measured acceleration and thecalculated change of orientation.
 11. A computer implemented method fordetecting a fall of a user, comprising: identifying a measurement ofacceleration and magnetic field associated with the user as a suspectedfall; and categorizing the suspected fall based on stored audio as oneof an activity of daily life (ADL), a confirmed fall, and aninconclusive event.
 12. The method of claim 11, wherein categorizing thesuspected fall based on the stored audio comprises an opt-out mechanismwherein specific sound patterns are used to assess the suspected fall asan ADL.
 13. The method of claim 11, wherein categorizing the suspectedfall based on the stored audio comprises an opt-in mechanism whereinspecific sounds characteristics are detected and employed to confirmthat a fall has happened.
 14. The method of claim 11, further comprisingcalculating a change of orientation of the user from the gyroscope, themagnetometer, and accelerometer that is more accurate than a change oforientation calculated from the magnetometer and accelerometer alone 15.The method of claim 11, further comprising receiving a notification andan optional further confirmation from a voice conversation with a callcenter or the user.
 16. The method of claim 14, further comprisingextracting at least one feature from the stored audio and at least oneof the measured acceleration and the calculated change of orientation.17. The method of claim 16, wherein: the at least one feature is aninter-signal dynamic property is based on relationships between audioenergy and physical movement; and the inter-signal dynamic property isone of elapsed time between acceleration and audio peaks andacceleration and rotation rate peaks.
 18. The method of claim 11,further comprising: transmitting the recorded audio and at least one ofthe measured acceleration and the calculated change of orientation to acloud computing system; and receiving a re-confirmation or change ofclassification from the cloud computing system based on the recordedaudio and at least one of the measured acceleration and the calculatedchange of orientation.
 19. A computer readable storage medium includinginstructions that, when executed by a processing system, cause theprocessing system to perform operations comprising: identifying ameasurement of acceleration and magnetic field associated with the useras a suspected fall; and categorizing the suspected fall based on storedaudio as one of an activity of daily life (ADL), a confirmed fall, andan inconclusive event.
 20. The computer-readable medium of claim 19,wherein the method further comprises: calculating a change oforientation of the user from the gyroscope, the magnetometer, andaccelerometer that is more accurate than a change of orientationcalculated from the magnetometer and accelerometer alone.